Stop installing setuptools and wheel on Python 3.12+ #218
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
edmorley
force-pushed
the
no-setuptools-wheel
branch
from
8581957 to
457539d
Compare
8 tasks
edmorley
force-pushed
the
no-setuptools-wheel
branch
from
457539d to
2f463eb
Compare
|
Please could I have review? :-) |
pradyunsg
reviewed
Jul 29, 2024
Yes, now that I have some bandwidth. |
Currently `get-pip.py` installs not only pip, but also setuptools and wheel by default, unless the `--no-setuptools` / `--no-wheel` (or `PIP_NO_SETUPTOOLS` / `PIP_NO_WHEEL` env vars) are used. This has historically been necessary, however, modern versions of pip will now fallback to `pyproject.toml` (PEP 517: [1]) based builds (which will default to a setuptools backend, and thus automatically install setuptools and wheel in the isolated build environment) if either setuptools is not installed (as of pip 22.1: [2]), or if wheel is not installed (as of pip 23.1: [3]). In addition, as of Python 3.12, the stdlib's `ensurepip` and `venv` modules no longer install setuptools, and only install pip ([4]). As such, it is now time for `get-pip.py` to stop installing setuptools and wheel by default on Python 3.12+, in order to: - Act as another small step towards `pyproject.toml` / PEP 517 based builds eventually becoming the pip default. - Improve parity with the behaviour of `ensurepip` / `venv` on Python 3.12+. - Allow `get-pip.py` to focus on its primary responsibility: bootstrapping Pip. Closes pypa#200. [1]: https://peps.python.org/pep-0517/ [2]: pypa/pip#10717 [3]: pypa/pip#11871 [4]: python/cpython#101039
edmorley
force-pushed
the
no-setuptools-wheel
branch
from
2f463eb to
def4aec
Compare
|
@pradyunsg Thank you for the review! I've made those changes. I wouldn't normally force push, but since this repo appears to exclusively use standard merges rather than squash and merge, I wanted to avoid a "Apply review feedback" type commit from ending up in the log on |
|
Testing with Python 3.12: And with Python 3.11: |
edmorley
added a commit
to heroku/buildpacks-python
that referenced
this pull request
Aug 7, 2024
Currently the buildpack performs a system site-packages install of not only pip, but also setuptools and wheel. This has historically been necessary for pip to be able to build source distributions (sdists) for packages that don't ship with compatible wheels. However: - Thanks to PEP 518, packages can now (and many already do) specify an explicit build backend using `[build-system]` in their `pyproject.toml`. The dependencies specified in that config (such as setuptools and wheel) will be installed by pip into an isolated and ephemeral build environment as part of the source distribution build process. Such packages therefore don't need/use globally installed setuptools/wheel versions. - As of pip v22.1, pip will now default to the isolated build environment mode (along with a fallback legacy setuptools build backend), if the setuptools package isn't installed globally. This means that packages that haven't yet migrated to a PEP 518 `pyproject.toml` build backend config can still build even if setuptools isn't installed globally. There are a small number of rarely used packages in the wild that aren't compatible with build isolation mode, however, these typically require more build dependencies than just setuptools, which means they wouldn't have worked with this buildpack anyway. As such, it's no longer necessary for us to install setuptools and wheel globally. This matches the behaviour of the `venv` and `ensurepip` modules in Python 3.12+, where setuptools and wheel installation has also been removed. And it also matches the default behaviour of Poetry too, whose `install --sync` command removes any implicitly installed packages in the current environment (other than pip). See: https://peps.python.org/pep-0518/ https://pip.pypa.io/en/stable/reference/build-system/ https://pip.pypa.io/en/stable/reference/build-system/pyproject-toml/#build-isolation pypa/pip#10717 python/cpython#101039 pypa/get-pip#218 astral-sh/uv#2252 GUS-W-16437776.
edmorley
added a commit
to heroku/buildpacks-python
that referenced
this pull request
Aug 7, 2024
Currently the buildpack performs a system site-packages install of not only pip, but also setuptools and wheel. This has historically been necessary for pip to be able to build source distributions (sdists) for packages that don't ship with compatible wheels. However: - Thanks to PEP 518, packages can now (and many already do) specify an explicit build backend using `[build-system]` in their `pyproject.toml`. The dependencies specified in that config (such as setuptools and wheel) will be installed by pip into an isolated and ephemeral build environment as part of the source distribution build process. Such packages therefore don't need/use globally installed setuptools/wheel versions. - As of pip v22.1, pip will now default to the isolated build environment mode (along with a fallback legacy setuptools build backend), if the setuptools package isn't installed globally. This means that packages that haven't yet migrated to a PEP 518 `pyproject.toml` build backend config can still build even if setuptools isn't installed globally. There are a small number of rarely used packages in the wild that aren't compatible with build isolation mode, however, these typically require more build dependencies than just setuptools, which means they wouldn't have worked with this buildpack anyway. As such, it's no longer necessary for us to install setuptools and wheel globally. This matches the behaviour of the `venv` and `ensurepip` modules in Python 3.12+, where setuptools and wheel installation has also been removed. And it also matches the default behaviour of Poetry too, whose `install --sync` command removes any implicitly installed packages in the current environment (other than pip). See: https://peps.python.org/pep-0518/ https://pip.pypa.io/en/stable/reference/build-system/ https://pip.pypa.io/en/stable/reference/build-system/pyproject-toml/#build-isolation pypa/pip#10717 python/cpython#101039 pypa/get-pip#218 astral-sh/uv#2252 GUS-W-16437776.
edmorley
added a commit
to heroku/buildpacks-python
that referenced
this pull request
Aug 7, 2024
Currently the buildpack performs a system site-packages install of not only pip, but also setuptools and wheel. This has historically been necessary for pip to be able to build source distributions (sdists) for packages that don't ship with compatible wheels. However: - Thanks to PEP 518, packages can now (and many already do) specify an explicit build backend using `[build-system]` in their `pyproject.toml`. The dependencies specified in that config (such as setuptools and wheel) will be installed by pip into an isolated and ephemeral build environment as part of the source distribution build process. Such packages therefore don't need/use globally installed setuptools/wheel versions. - As of pip v22.1, pip will now default to the isolated build environment mode (along with a fallback legacy setuptools build backend), if the setuptools package isn't installed globally. This means that packages that haven't yet migrated to a PEP 518 `pyproject.toml` build backend config can still build even if setuptools isn't installed globally. There are a small number of rarely used packages in the wild that aren't compatible with build isolation mode, however, these typically require more build dependencies than just setuptools, which means they wouldn't have worked with this buildpack anyway. As such, it's no longer necessary for us to install setuptools and wheel globally. This matches the behaviour of the `venv` and `ensurepip` modules in Python 3.12+, where setuptools and wheel installation has also been removed. And it also matches the default behaviour of Poetry too, whose `install --sync` command removes any implicitly installed packages in the current environment (other than pip). See: https://peps.python.org/pep-0518/ https://pip.pypa.io/en/stable/reference/build-system/ https://pip.pypa.io/en/stable/reference/build-system/pyproject-toml/#build-isolation pypa/pip#10717 python/cpython#101039 pypa/get-pip#218 astral-sh/uv#2252 GUS-W-16437776.
|
Please could I have a re-review? :-) |
pradyunsg
approved these changes
Aug 12, 2024
|
Please may someone merge this, now that it's had review (and time for anyone else to comment/review/object)? :-) |
This was referenced Aug 21, 2024
|
Thanks for the poke @edmorley! I've merged this but we can revert this if someone raises concerns around this. |
Closed
|
Thank you :-) As a reminder to anyone coming across this PR who wants to return to the old behaviour, you can update your |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Currently
get-pip.pyinstalls not only pip, but also setuptools and wheel by default, unless the--no-setuptools/--no-wheel(orPIP_NO_SETUPTOOLS/PIP_NO_WHEELenv vars) are used.This has historically been necessary, however, modern versions of pip will now fallback to
pyproject.toml(PEP 517) based builds (which will default to a setuptools backend, and thus automatically install setuptools and wheel in the isolated build environment) if either setuptools is not installed (as of pip 22.1), or if wheel is not installed (as of pip 23.1).In addition, as of Python 3.12, the stdlib's ensurepip and venv modules no longer install setuptools, and only install pip.
As such, it is now time for
get-pip.pyto stop installing setuptools and wheel by default on Python 3.12+, in order to:ensurepip/venvon Python 3.12+.pyproject.toml/ PEP 517 based builds eventually becoming the pip default. (Cross-reference: Default to isolated, pyproject.toml-based builds pip#9175)get-pip.pyto focus on its primary responsibility: bootstrapping Pip.Users can still request that setuptools and/or wheel are installed on Python 3.12+, by adding them to the
get-pip.pyinvocation like so:$ python get-pip.py setuptools wheelCloses #200.